Digital images and video have become increasingly common in entertainment, research, education, business and broadcasting. This is due to the many benefits that result from digital representation of images and video. These benefits include lower overall costs of capturing, storing, processing, and transporting images in digital form. In addition, digital formats allow flexible manipulation of images including the formation of multilayered images with relative ease.
Multilayered digital images are formed by compositing two or more digital images. Compositing typically involves overlaying an overlay image (called the foreground plane), onto a background image (called the background plane). Compositing is detailed, for example in T. Porter and T. Duff, “Compositing Digital Images”, Proceedings of SIGGRAPH '84, 18 (1984).
Pixels in the foreground plane may be opaque or semitransparent. When compositing, if a pixel in the foreground plane is opaque then the corresponding pixel in the background plane is overwritten. Conversely, if a pixel in a foreground plane is completely transparent, then the corresponding pixel in the background plane becomes the new pixel of the newly formed image.
If a foreground pixel is semitransparent then both the foreground pixel and the corresponding background pixel contribute to the newly formed image pixel in proportion to a transparency value associated with the foreground pixel. Pixels of the foreground and background images that are at the same pixel location are linearly combined in accordance with the transparency value of the foreground image to form pixels of the new composited image.
The transparency value of a semitransparent foreground image pixel is sometimes called the alpha value (denoted by a), and thus digital composition of images is sometimes referred to as alpha-blending.
As will be appreciated, more images/planes may be overlaid onto the newly formed image. The already formed image becomes the new background image, to be combined with a new foreground image in a new composition process.
Examples of multilayered images formed by digital composition include video images with some overlay graphics, picture-in-picture (PiP) video frames, video frames displaying menu options and the like.
The popularity of multilayered digital images has led to the adoption of newer video formats that provide multiple planes of video data. Examples of these newer formats include the Blu-ray format, and the high definition digital versatile disc (HD DVD) format. In Blu-ray, a main video stream is accompanied by a presentation graphics stream and an interactive graphics stream. These graphics streams contain overlay information associated with subtitles, menus, picture-in-picture (PiP) video, command buttons and other types of graphics overlays. Similarly, HD DVD video format includes several streams associated with a main video plane, a background plane, a secondary video plane, a subtitle graphics plane, an application graphics plane, and a cursor plane.
Each of these streams may be compressed. As a result, a video source such as a Blu-ray player or a computing device, typically first decodes each compressed stream to form planes, and then composites corresponding planes to form output images for display.
Composited video images are transmitted to a display. Typical displays include liquid crystal displays (LCD), CRT monitors, digital television (DTV) sets and the like. Some displays may have their own image processing capabilities such as de-interlacing, and color brightness or contrast adjustments.
Unfortunately however, receiving already composited frames restricts the type of image processing that can be applied at a display. In addition, it limits the effective use of existing video processing capabilities of displays that may often be more powerful than the capabilities of typical video sources.
Accordingly, there remains a need for methods and devices that allow flexible and effective processing of multilayered video data received in displays.